Windows EC2 パフォーマンス問題を AWS サポート自動化ワークフローで診断してみた

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ec2:DescribeInstances",
                "ssm:DescribeAutomationExecutions",
                "ssm:DescribeInstanceInformation",
                "ssm:GetAutomationExecution",
                "ssm:ListCommands",
                "ssm:ListCommandInvocations",
                "ssm:SendCommand",
                "s3:GetAccountPublicAccessBlock"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetEncryptionConfiguration",
                "s3:GetBucketPublicAccessBlock",
                "s3:GetBucketPolicyStatus",
                "s3:GetBucketAcl",
                "s3:PutObject",
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::xxxxxxxxxxxx",
                "arn:aws:s3:::xxxxxxxxxxxx/*"
            ]
        }
    ]
}

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "",
            "Effect": "Allow",
            "Principal": {
                "Service": "ssm.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetObject",
                "s3:PutObject",
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::xxxxxxxxxxxx",
                "arn:aws:s3:::xxxxxxxxxxxx/*"
            ]
        }
    ]
}

The instance has enough space to dump impacting process. Currently available space: 55.66 GB is more than minimum required Space. Proceeding further to capture dump.

[PASSED] Dump captured successfully for the process: CPUSTRES64 with ID: 5276. 
The dump file will not be deleted by this automation, please manually delete it after analysis. 
The dump file location: C:\ProgramData\Amazon\SSM\TroubleshootWindowsPerformance\xxxxx-xxxx-xxxx-xxxx-xxxxxxx\CPUSTRES64_5276_06-18-40.dmp

The instance has enough space to capture performance logs.
WPR capture process is in 'Stopped' state.
Data Collector Set TroubleshootWindowsPerformance_xxxxx-xxxx-xxxx-xxxx-xxxxxxx was not found.
Attempting to create Performance monitor Data Collector Set TroubleshootWindowsPerformance_xxxxx-xxxx-xxxx-xxxx-xxxxxxx......
Data Collector Set TroubleshootWindowsPerformance_xxxxx-xxxx-xxxx-xxxx-xxxxxxx created successfully.
Attempting to start Performance monitor Data Collector Set TroubleshootWindowsPerformance_xxxxx-xxxx-xxxx-xxxx-xxxxxxx......
Data Collector Set TroubleshootWindowsPerformance_xxxxx-xxxx-xxxx-xxxx-xxxxxxx started successfully.
Current CPU usage is '79.91'% and Memory usage is '51.29'%
Not both CPU and Memory usage are over 95% at this moment hence continue to capture WPR log.
Starting Windows Performance Recording (WPR) capture process.
Stopping WPR capture process.
WPR capture process is in 'Stopped' state.
The Data Collector Set TroubleshootWindowsPerformance_xxxxx-xxxx-xxxx-xxxx-xxxxxxx is currently generating logs.
The Data Collector Set TroubleshootWindowsPerformance_xxxxx-xxxx-xxxx-xxxx-xxxxxxx has finished generating logs and is currently in 'Stopped' state.
Attempting to delete Data Collector Set TroubleshootWindowsPerformance_xxxxx-xxxx-xxxx-xxxx-xxxxxxx.
Data Collector Set TroubleshootWindowsPerformance_xxxxx-xxxx-xxxx-xxxx-xxxxxxx deleted successfully.

[PASSED] Performance logs are captured successfully inside the folder: C:\ProgramData\Amazon\SSM\TroubleshootWindowsPerformance\xxxxx-xxxx-xxxx-xxxx-xxxxxxx 
The captured log files will not be deleted by this automation, please manually delete it after analysis.

Top 5 Processes which consumed most CPU in percentage as below. If you see a percentage higher than 100 that means the process is using more than one CPU core.
Process                   Counter         Min %           Max %           Avg %          
CPUSTRES64                Processor       132.00          149.00          143.00         
System                    Processor       0.00            24.00           0.31           
ssm-document-worker       Processor       0.00            12.00           0.06           
powershell                Processor       0.00            8.00            0.05           
WmiPrvSE##1               Processor       0.00            5.00            0.08           

Top 5 Processes which consumed most WorkingSet64 memory as below (in MB):
Process                   Counter         Min MB          Max MB          Avg MB         
MsMpEng                   WorkingSet      157.00          189.00          160.00         
svchost##3                WorkingSet      181.00          181.00          181.00         
msedge                    WorkingSet      171.00          171.00          171.00         
explorer                  WorkingSet      170.00          170.00          170.00         
dwm##1                    WorkingSet      120.00          121.00          120.00         

[PASSED] Performance logs summary completed.
For detailed analysis of other metrics such as Disk, Memory, Network Interface, TCPV4, IPV4 and UDPV4, please check the file C:\ProgramData\Amazon\SSM\TroubleshootWindowsPerformance\xxxxx-xxxx-xxxx-xxxx-xxxxxxx\analysis_output.log 
You can also analyze the captured Windows Performance Recorder logs as well as Windows Performance monitor logs captured in the same directory.
The analysis_output.log file will not be deleted by this automation, please manually delete it after analysis.

The log folder has been compressed to C:\ProgramData\Amazon\SSM\TroubleshootWindowsPerformance\Zip\xxxxx-xxxx-xxxx-xxxx-xxxxxxx\TroubleshootWindowsPerformance_i-xxxxxxxxxxxx_xxxxx-xxxx-xxxx-xxxx-xxxxxxx.zip. Uploading the compressed archive to the Amazon S3 bucket...

[PASSED] Log files uploaded to S3 bucket xxxxxxxxxxxx with key C:\ProgramData\Amazon\SSM\TroubleshootWindowsPerformance\Zip\xxxxx-xxxx-xxxx-xxxx-xxxxxxx\TroubleshootWindowsPerformance_i-xxxxxxxxxxxx_xxxxx-xxxx-xxxx-xxxx-xxxxxxx.zip. 
The captured log files are still placed in the instance, they won't be deleted by the automation and will need to be deleted manually after troubleshooting.

----------ERROR-------
C:\ProgramData\Amazon\SSM\InstanceData\i-xxxxxxxxxxxx\document\orchestration\xxxxx-xxxx-xxxx-xxxx-xxxxxxx\
awsrunPowerShellScript\0.awsrunPowerShellScript\_script.ps1 : [FAILED] An error occurred when trying to upload C:\Progr
amData\Amazon\SSM\TroubleshootWindowsPerformance\Zip\xxxxx-xxxx-xxxx-xxxx-xxxxxx\TroubleshootWindowsPerformanc
e_i-xxxxxxxxxxxx_4xxxxx-xxxx-xxxx-xxxx-xxxxxx.zip User: arn:aws:sts::xxxxxxxxxxxx:assumed-role/xxxxx/i-xxxxxxxxxxxx is not authorized to perform: s3:PutObject on resource: "arn:aws:s3:::xxxxxxxxx/TroubleshootWindowsPerformance_i-xxxxxxxxxxxx_xxxxx-xxxx-xxxx-xxxx-xxxxxxx.zip" because no identity-base
d policy allows the s3:PutObject action. Please check and upload the logs manually after accessing the instance.
    + CategoryInfo          : NotSpecified: (:) [Write-Error], WriteErrorException
    + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,_script.ps1

failed to run commands: exit status 1

./TroubleshootWindowsPerformance_i-xxxxxxxxxxxx_xxxxx-xxxx-xxxx-xxxx-xxxxxxx
├── CPUSTRES64_5276_06-18-40.dmp
├── analysis_output.log
├── i-xxxxxxxxxxxx_EC2Rescue_06-34-44.zip
├── i-xxxxxxxxxxxx_WPR_06-18-51.etl
├── i-xxxxxxxxxxxx_perfmon_06-18-55.blg
├── netsh_ipv4_ipstats.txt
├── netsh_ipv4_tcpconnections.txt
├── netsh_ipv4_tcpstats.txt
├── netsh_ipv6_ipstats.txt
├── netsh_ipv6_tcpstats.txt
├── report.html
├── report.xml
├── report.xsl
├── rules.log
├── sysinfo.txt
├── tasklist.txt
└── tasklist_svc.txt